IAES International Journal of Robotics and Automation (IJRA) 
Vol. 12, No. 4, December 2023, pp. 352~364 
ISSN: 2722-2586, DOI: 10.1159 1/ijra.v12i4.pp352-364 g 352 


Robotic inspection and automated analysis system for advanced 


manufacturing 


Maged Mikhail, Karen Abad 


Department of Engineering Technology, College of Technology, Purdue University Northwest, Hammond, IN, United States 


Article Info 


ABSTRACT 


Article history: 


Received Apr 20, 2023 
Revised May 28, 2023 
Accepted Jun 4, 2023 


Keywords: 


Advanced manufacturing 
Automated system 
Programmable logic controller 
application 

Robot integration 

Vision system 


Companies have developed various systems to improve their processes. 
These processes’ focus has been to produce more quantity in less time. To 
accomplish this task, it is important to also consider defects. Defective 
products can cause delays in the production line, rework, and the loss of 
money, time, and resources. This project focused on developing an 
integrated inspection system. Previous research has been done regarding 
types of vision systems, in-line inspections, and feedback data collection. A 
programmable logic controller (PLC) was used to control when the conveyor 
belt starts and stops. When the object has reached a certain position, the 
camera detects if the object passed or failed the process. If the object fails, 
the robot will pick up the bottle and take it out of the line. Human-machine 
interface (HMI) was also integrated, which shows how many bottles have 
passed and failed with a light that will indicate if a certain object has passed 
or failed. Feedback from the inspection process can help solve potential 
issues from different machines and processes. The testbed was designed, 


integrated, and tested in the paper to perform a feedback analysis for the 
production line. The setup consisted of MicroLogix PLC, Fanuc robot LR 
Mate 200iD and Cognex camera. 
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1. INTRODUCTION 

Inspection is a required process in many industries. Among the industries where the inspection was 
used as a required process is power generation with a total of 23.5%, followed by aerospace, military, and 
defense with 23.1%. Results from previous research showed that visual inspection errors range between 20% 
to 30% being caused by human error or limited space [1]-[3]. A different study proved that manual 
inspection is only 85% efficient [4]-[6]. Human inspection is not one of the most reliable methods of 
inspection since it can lead to less consistent results due to human error and different levels of expertise [3]. 
Using vision systems can provide a more reliable system with better inspection results. Automating vision 
systems also provides cost savings to industries. Inspection by humans is estimated to be only 10% of the 
workforce which is only around 400,000 jobs, meaning an estimated $8 billion is dedicated to industrial 
inspection [7], [8]. Investing in vision systems will improve the quality of the products offered to the 
consumer while reducing costs. 

The quality inspection process does come with some benefits. The benefits mentioned by the 
participants include higher quality products, lower costs, reduction in recalls, regulatory compliance, 
improved production throughout, less rework, and less waste [9], [10]. The quality inspection can be 
performed before, during, and after production. During production, other processes occur such as labeling or 
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capping. Detecting defects before a product is manufactured is an important step. It can help employees to 
implement corrective actions before delivering to the consumer. It can help reduce rework, improve 
consumer dissatisfaction, and prevent disruption in the manufacturing line and delay of products to customers 
[11]-[13]. 

Feedback from an inspection process can help the operators detect a potential error in one of the 
machines. Enhanced self-inspection is an inspection type that uses devices to automatically detect defects by 
giving the operator feedback, which helps to prevent defects and achieve 100% inspection [14]-[16]. This 
analysis can be made by having a range of failed items and analyzing which number it will consider 
abnormal. Having separate inspections from different processes can facilitate the process of looking at which 
process is causing defects and which machine will be responsible for causing these actions. 

The project focused on the development of an integrated vision system to collect data. To develop 
this concept, bottles were used to detect defects in the label and in the bottle caps. The system was controlled 
using an HMI that had “start,” “stop,” “reset,” and indicator lights and provided information about how many 
bottles passed and failed the inspection. The bottle stops in front of a camera so that it can send feedback to 
the PLC, and this can start collecting the information. The system also has a numeric control where the user 
can specify the percentage of efficiency that it requires and if the system did not meet the efficiency, it 
activates an alarm in the HMI notifying the user that improvements are required in the system during the 
process. 


2. RESEARCH METHOD 
2.1. Statement of the problem 

To meet customer expectations, quality control is important in the industry. Human operators 
perform most inspections. This method is not one of the most accurate due that human operators may face 
fatigue while performing inspections all day long [8]. An inaccurate inspection can bring consequences in the 
industry such as delays, rework, and waste of time, money, and resources. The benefit of using automated 
vision inspections is that data can be collected from the processes which can be analyzed and used to improve 
a process or a machine’s functioning. 


2.2. Project objective 

The main objective of this project is to develop a feedback automated system that can gather data 
from an integrated inspection process. Results will provide insight into how this can be applied in the 
production line and how it will improve the quality of the process. 


2.3. Definitions 
2.3.1. PLC 

PLC is a programmable logic controller that uses programmable memory to store different 
instructions and to implement functions such as timing, logic, counting, sequencing, and arithmetic. These 
functions are implemented to control processes and different machines. PLCs are similar to computers. 
However, these are designed for industrial environments and control tasks. The components of a PLC consist 
of a power supply, a processor unit, input and output interfaces, a communications interface, and a 
programming device as shown in Figure 1. 
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Figure 1. PLC architecture [10] 
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2.2.2. Sinking and sourcing 

Devices that communicate with the PLC can be wired to the DC in two different ways, which are 
sinking and sourcing [10]. During sourcing, the input module becomes the current source, meaning that the 
device receives current from the input from positive to negative. During the sinking, the input module 
becomes the sink of the current, which means that the device supplies current to the input. 


2.2.3. Sourcing sinking output 
Sourcing occurs when the current flows from the output to the load. On the other hand, sinking 
occurs when the current flows from the load to the output as shown in Figure 2. 


Output Output 
module module 


Output load p Output load 


Figure 2. Sourcing output diagram 


2.2.4. Teach pendant 

Teach pendant is an interface between the operator and the application tool software. The teach 
pendant is connected to the controller’s circuit board. Teach pendant is used to create programs, jog the 
robot, and check the status, operation, and test execution. The teach pendant has three main switches. The 
enable switch is used to enable or disable the teach pendant as shown in Figure 3. The emergency stop button 
is used to stop the robot immediately when it is pressed. The Deadman switch is used to enable the robot’s 
motion if the switch is not gripped properly; it will not let the robot move [17]-[19]. 


Emergency 
stop button 


Teach pendant N TN a nae A 
enable switch Oy — —A—T) 


[se T) 
| 


(21212101703) 
gogogog; 


Figure 3. Fanuc robot teaches pendant 


2.2.5. Operator panel 

The operator panel can perform different functions such as starting a program and releasing an 
alarm state. The switches that the operator’s panel has are an alarm release button that is responsible for 
releasing an alarm state, a start button that is used to start a currently selected program, an emergency stop 
that is used to stop the robot immediately, and the three-mode switch that is used to choose from three 
different operation modes that will be suitable for each different environment as shown in Figure 4. 


2.4. Delimitations 

The project took place at the Commercialization and Manufacturing Excellence Center (CMEC). 
Robots and industrial equipment are available at this facility. The scope of the project was collecting data 
from inspection. The scenario created to show this concept consists of an inspection where the data results 
are shown on the HMI. This indicates the number of bottles that passed and failed the inspection. The system 
prevents the products from continuing with the defects and it helps to notify the operators so that actions can 
be taken immediately. 
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Mode Switch 
(Three mode switch) Start (Green) Emergency Stop 


Figure 4. Robot operator panel [11] 


2.5. Research methodology 

Previous research was conducted to verify all devices were compatible. Since the project is a 
pass-and-fail inspection, an Insight 2,000 camera was selected because it can perform inspection and send 
information to the camera outputs. Looking at the conveyor belt parameters, PowerFlex 525 was selected as 
it met the motor requirements. Different available wiring diagrams were used to properly connect the 
devices. The Application of Industrial Robots for Advanced Manufacturing (TECH 58100) was a helpful 
reference in starting this project since many application capabilities were learned regarding the Fanuc 
operation and integration. It guided the integration design and capability. This class also introduced vision 
systems which were a helpful guide for the inspection system. The Graduate Professional Practice (TECH 
69500) helped students to get a view of the industrial world and its needs in many industries. It also helped 
with the project planning and development process. The lean six sigma certification classes were a good base 
for quality, continuous improvement, and statistical analysis. Those classes guided the data collection 
analysis and how this can improve quality in the workplace. The Analysis of Research in Industry and 
Technology course (TECH 64600) helped with the proposal design and with analyzing results from previous 
projects. Literature reviews from previous projects also were a helpful guide for this project. 


2.5.1. Fanuc robot 

Fanuc Robot LR Mate 200iD was selected to be in this project since it’s available in the lab. Fanuc 
robot has six controlled AC servo motors; the maximum payload for this particular robot is 4 kg, a 
mechanical weight of about 20 kg, a 0.01 mm repeatability error, and the maximum reach of the arm is 
550 mm. Figure 5 shows the operating space for this robot [20], [21]. 
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Figure 5. Fanuc robot specifications 
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.5.2. Insight 2000 camera 
A Cognex industrial camera was used for this project. This camera has a speed of 75 frames per 
second and is powered with the same PLC power supply. The insight camera is shown in Figure 6. 


Figure 6. Insight 2000 camera 


2.5.3. Conveyor belt 
The conveyor belt motor model is a VWDM3538. It has two specifications. 

- Network Communication Design: The communication used for this project is Ethernet/IP. Devices are 
connected to a common switch. The table below shows the IP address used for each device. The subnet 
mask is 255.255.255.0 and the default gateway is 192.168.1.1. 

- Integration of the PLC with the robot: Integrating the PLC with the robot allows better control of the 
operation because everything can be controlled by the PLC program. Integrating everything into a 
common program also helps to create synchronization so that when machines stop, the next one can start 
operating. 

Fanuc robots can be connected to peripheral devices by installing a connector converter board. The converter 

is compatible with the peripheral device control interface as shown in Figure 7. Figure 8 shows how the 

robot’s digital inputs and outputs are mapped to the PLC. 
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Figure 7. Fanuc Robot connector 
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Figure 8. Robot IO mapping 


IAES Int J Rob & Autom, Vol. 12, No. 4, December 2023: 352-364 


IAES Int J Rob & Autom ISSN: 2722-2586 o 357 


2.6. Run the Fanuc robot 

The operator panel should be adjusted to move the robot remotely by switching the mode switch to 
AUTO. Table 1 explains the different modes and how the teach pendant, operator’s panel, and peripheral 
devices operate when is local or remote. 


Table 1. Robot operation mode 


Mode Teach pendant enable switch _ Remote switch _ Teach pendant 
AUTO Mode ON Local Not allowed 
Remote Not allowed 
OFF Local Not allowed 
Remote Not allowed 
T1, T2 Mode ON Local Allowed to start 
Remote Allowed to start 
OFF Local Not allowed 
Remote Not allowed 


2.6.1. Integration of the HMI 

HMI is used to display real-time information to the user about the inspection results. In this case, it 
has indicator lights showing when the system passes or fails as shown in Figure 9. It has pushbuttons that 
allow the starting and stopping of the system just by pressing them. It also shows the total number of items 
that passed and failed. The PanelView Plus 600 is connected to the power supply 24VDC. The Ethernet port 
communicates with the Ethernet switch so that the program can be transferred to the HMI. 


Figure 9. HMI display 


2.6.2. Configuring the powerful 

The IP address needs to be configured first before starting to communicate with the system through 
Ethernet. The parameters for the IP address, subnet mask, and gateway were configured from the drive’s 
keypad. It was made sure the parameters in P046 are set to 5 Ethernet/IP and P047 is set to 15 Ethernet/IP. 

First, the C128 needs to be 1 so that the IP address can be configured with the driver’s keypad. 
Researchers then set the IP address values using the parameters from C129 through C132. After that, the 
subnet mask was configured so the parameters can be set from C133 to C136. Finally, to set the gateway 
address, the parameters are set from C137 through C140. After changing the values, researchers reset the 
adapter by power cycling the drive so the values can take effect. 


2.6.3. PLC programming 

After all the inputs and outputs were wired and assigned to perform different functions, a program 
was created using the address for each device. The PLC program was designed for the user to enter the 
number of bottles that are going to be inspected and the percentage of efficiency that the operator expects the 
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inspection to meet. If failed bottles exceed the minimum acceptable range, then an alarm is activated as 
shown in Figure 10. 

The percentage of minimum failed items is taken based on the number of bottles, which is N7:5, and 
the efficiency percentage specified by the user is N7:3. Taking this percentage, the total number of bottles 
allowed to fail is obtained and is represented by N7:7. The result obtained from the calculation is sent to 
N7:7. After the result is obtained, it is compared with the accumulated value of failed bottles for cap and 
label defects. If the value is greater than or equal to the calculated value, it will send an alarm to the operator 
which can help the operator to make improvements to the system or the machine. 
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Figure 10. Sample of PLC program 


2.6.4. Additional requirements specified by the user 

This program is designed for use in cases of continuous defects. If the program detects the same 
kind of defects repeatedly, it will send an alarm so that respective actions can be taken. It might be a problem 
involving the machine. 

The program has a timer that, if the accumulated value reaches the preset value, will send an alarm 
for repeated defects. The timer will reset when inspections that pass are produced. The timer preset value is 
the destination of a calculation created based on the inspection time and the number of defects that the 
company specified as the maximum that can fail in a row. The same logic is applied to both defects. In the 
test, three bottles with different label defects were used to verify the program operation. After specifying the 
minimum number of bottles allowed to fail, the user will get an alarm after three bottles have failed the 
inspection. The cap defects were also included in the testing. The defects used for the caps were two darker 
colors and no cap present on the product. 
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2.7. HMI designing and programming 

Before programming the HMI, it is required to assign the path from which the tags are dragged into 
the functions for the HMI programming. That step is accomplished by going to the communications setup 
under FactoryTalk Configuration setup. Momentary push buttons were used to start, stop, and reset the 
conveyor belt as shown in Figure 11. The address assigned to the start pushbutton is B3:0/1, to stop the 
B3:0/0 and reset the B3:3/3. Two indicator lights were used to display when a bottle passed or failed. The 
addresses assigned to the lights are the ones coming from the camera; the I: 0/1 for a pass and the I:0/0 for a 
fail. On the right-side screen of the HMI, the information feedback for the bottle inspection is displayed. The 
total bottles that passed and failed the cap inspection as well as the total bottles that passed and failed the 
label inspection are listed. 


AP INSPECTION 


NN* 


NN* 


LABEL INSPECTIO 


NN* 


NN* 


Figure 11. HMI display layout 


After the program has been created, it is transferred to the HMI by creating a runtime application 
using the version that is compatible with the HMI. In this case, the version in the HMI is five so the runtime 
created was 5.0. The file created is used in the transfer utility to download to the HMI. 


2.7.1. Requirements specified by the user 

The HMI consists of numeric controls and numeric indicators for the number of bottles. The HMI 
has numeric fractions that the user can use to monitor repeated failures before the alarm is activated. The 
HMI has control indicators so that the efficiency percentage can be specified as well as the number of bottles 
that are going to be inspected as shown in Figure 12. A fraction was included to visualize the number of 
bottles failing compared with the total for both inspections. Two multistate indicators were included to 
indicate when an alarm is on. 


2 ADDITIONAL USER SPECIFICATION... |->-||-©-] faia) 
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NO ALARM 


NO ALARM MAIN 


Figure 12. Updated HMI 
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2.7.2. Camera setup 

The insight explorer is required to configure it to get data from the camera and send it to the PLC. 
The first application step is connected and consists of selecting the in-sight sensor that needs to be connected. 
On the same page, a saved program can be opened to continue with previous work. The second step is to set 
up the image. In this screen, the trigger type is selected. Since the camera is connected to the PLC through 
wiring I/O, the trigger is the camera. The delay chosen is one second so that the camera has time to get a 
good focus on the object as shown in Figure 13. The lighting parameters are adjusted according to the object 
and the lighting of the room. 

The next application step is the setup tools to locate and inspect the part. Different tools can be used 
to get an inspection result. The pattern tool is used for this application. Pattern 1 specified is to record the cap 
requirements as shown in Figure 14. If the cap of the bottle does not have the same color as the specified 
color, it fails the inspection. 


Dla Sus ae Cat 


In-Sight Network x) | In-Sight Files 
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Gp is2000-130_44ccc0 


Figure 14. Locate part 1 in-sight explorer application step 
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2.7.3. Configure I/O 

The next step is to configure results for the input and output. The two outputs that are connected 
from the PLC output to the camera inputs were HSOUT 0 for fail and HSOUT | for a pass. The results of the 
inspection can be seen in the right window. If the inspection fails, the HSOUT 0 will have a value of 1 and if 
the inspection passes, the HSOUT2 will have a value of 1 as shown in Figure 15. 


User Data 


0 \KSOUTO Jod Resutt | Pattern 1 Fae 
SOUT 1 Jod Result | Pattern, 1 Pass 
MSOUT 2 Job Result | Pattern Lail 
HSOUT 3 Jod Result | Pattern 2Past 
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LEO 4 Red) Red LED Jod Result | Joofail 


Figure 15. Inputs/outputs in-sight explorer 


2.7.4. Testing application 

Many scenarios were conducted to validate the accuracy of the system. These scenarios had many 
combinations and shuffling between good and bad labels and good and bad caps as shown in Figure 16. The 
good label and good cap are shown in Figure 17. Finally, the robot gets the information from the camera via 
PLC and sorts the bottles into two different bins. One bin is for good bottles and one is for bad bottles as 
shown in Figure 18. 


Figure 16. Failed inspection due to missing a cap or a bad label 
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Figure 17. Passed inspection 


Figure 18. Automated sorting station 


2.7.5. Historian chart 

The last design implemented for the inspection system consists of a chart that provides real-time 
results on the number of bottles that have passed or failed the inspection as shown in Figure 19. The range 
specified for this application is 10 but the user can change these values depending on the total number of 
bottles. The pens used for the charts were assigned a tag that counts the total accumulated value for each 
condition. This screen also has a button to display previous data results. Having this data can help to analyze 
the different conditions and it is easier for the user to identify when something seems out of normal 
operation. 
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Figure 19. Historian chart for inspection feedback 
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3. RESULTS AND DISCUSSION 

Creating different programs in the robot to sort the defects helped to visualize the defect that was 
detected. In an industrial scenario, sorting the defects will reduce the time it will take to rework a certain 
item; for instance, implementing an automated cart at the end of the line will pick up the defects and 
distribute them for rework. The program had a delay when trying to run from the HMI; in an industrial 
setting, a SCADA interface can be used which provides a faster response. The camera needed some 
calibration and testing to make sure it recognized when something passes or fails. After changing some 
parameters, the camera was able to get accurate results. In an industrial scenario, the camera needs to be 
adjusted depending on the items to be inspected and the lighting of the room. 

The program designed for the user to select data specifications was successfully tested and the 
alarms will turn on depending on said specifications. During the test, the efficiency rate was set to 30% and 
the number of bottles used was three, so the alarm turned on after two bottles had failed the cap inspection. 
Before the user can input data, it is required to know the process, the machine’s normal operation, and the 
defect rate caused by the system. After these values have been obtained, the efficiency can be inserted in the 
screen and anything that is beyond the normal will send an alarm. This design will help industries to find a 
potential problem related to a specific inspection process. 

Alarms were created to notify the user whenever some user specification is out of range. The alarm 
displays a message regarding the problem so that an operator can act and analyze the situation if something is 
wrong with the machine or the process. The alarm appears even if the user is looking at another screen. Once 
the alarm is acknowledged, it will automatically close the pop-up notification. 

The last design made was the historian chart which was configured to display the total number of 
bottles that passed and failed due to an issue with the bottle’s cap or label. The chart displays real-time data, 
and a user can use the left arrow to look at previous data information. The chart can be configured according 
to the number of bottles used in the application and the variables can be configured according to the user’s 
needs. In the first test, the chart showed high values and the data results were at the bottom. After configuring 
the chart from 0 to 10, the chart got better visibility. The chart was successfully tested with different defects 
and values which were updated immediately in the chart. 


4. CONCLUSION 

This process was controlled by a PLC. A ladder logic program was designed for this operation and 
to run the machines. Their I/O were connected to the PLC I/O allowing them to include this information in 
the program. Multiple programs were used to configure the functioning of devices for this application. The 
use of wiring diagrams was useful to connect all the components to the PLC. The paper contains the 
connection made to each port of the PLC describing which device is connected. It also contains the IP 
addresses used for each device and how it will be included in the PLC program. The HMI receives all 
information from the PLC process and displays all information needed for the operator to use for feedback 
and to continue improvement. The screen also has indicator lights to show when the inspection failed or 
passed. The project results can be implemented in an industrial environment by automating all vision systems 
and having a common network where all the data can be seen from different machines. Analyzing the data 
and creating specifications for the system can help visualize when something is not working properly, or it 
can help to identify a potential problem related to the machines or the process. 
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